The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
Generally, the most currently existing commercial video compression methods and their apparatuses effectively remove temporal redundancies which are present in natural videos by using a block motion estimation/compensation method and an apparatus thereof. Such a block motion estimation/compensation method is based on an assumption that all the pixels in a block basically have the same motion, and performs a prediction/reconstruction on the respective block pixels of the subject current image to be encoded by using previously compressed and transmitted image(s). Propelled by the simple but efficient basic assumption, the block motion compensation has a little motion parameters to encode and transmit, and thereby makes a significant contribution to improving the compression efficiency of video data. However, this method involves blocking artifacts caused by a lack of conformity that some pixels (normally at block boundaries) in the blocks do not conform to a basic model, wherein the blocking artifact means unsightly latticed artificial coding errors observed at the respective block boundaries. In order to solve the blocking artifacts, many conventional techniques have become available, and the overlapped block motion compensation is one of them.
The technique of overlapped block motion compensation is performed in motion reconstructions for the respective blocks by a weighted addition of the reconstructed current location pixels based on motions of adjacent blocks to reconstructed pixels in a current block as a way of reflecting the motions of the adjacent blocks in the motion reconstruction of the current block, whereby errors in the motion reconstruction at the block boundaries are effectively and significantly reduced. However, in the case where the motions of the adjacent block are significantly deviated from the current block, the motion reconstruction by the overlapped block motion compensation is known to show a greater deterioration than the typical block motion reconstruction or cause blurring artifacts which blur edge information of the blocks. This phenomenon is often called over-smoothing problem, which occurs when the relatively correctly predicted pixel values at the block boundaries under the well satisfied assumption with the block motion estimation method undergo a weighted addition to pixels in the adjacent blocks having a critical error caused by the overlapped block motion reconstruction. Generally, the reason for showing a significant difference between motions of two adjacent blocks is because the most of the pixels in the respective blocks have disparities in actual motions which are caused by the different imaged objects of possibly different dynamic natures within the video. Therefore, such over-smoothing problem generally occurs to the in-video objects at their edges which happen to be extra important video information, and thus resolving the over-smoothing problem is important in further improving the video compression performance and obtaining an even higher quality reconstructed video.
To solve such over-smoothing problem, there exist prior arts in the form of various techniques that mostly address the motion reconstruction of the respective blocks by adaptively applying, via selecting between, the overlapped block motion compensation and the typical block motion compensation. According to Ji Zhongwei, Jiang Wenjun, and Zhu Weile (“Wavelet-based video coding using adaptive overlapped block motion compensation”, in Proc. ICCCS'02, 29 Jun.-1 Jul. 2002, vol. 2, pp. 1090-1093), encoding is performed as the respective blocks are tried with the overlapped block motion compensation and the conventional block motion compensation, and then mean-squared errors for the respective methods are compared to select one that has a lower value. Though this technique could have attained an improved motion reconstruction performance by way of time-processing always in a method with less errors in motion compensation, a decoder is unable to reproduce the mean-squared errors for the respective methods which the encoder used in its selection, necessitating transmission of a form of additional information as to the method for motion-reproducing the respective blocks and the consequent bit rate of the information may pointedly deteriorate the overall compression performance.
There are various types of prior arts to perform an adaptive overlapped block motion compensation while handling the problematic increase of bit rate with the extra information transmission, and firstly Tien-ying Kuo and C.-C. Jay Kuo (“A hybrid BMC/OBMC motion compensation scheme”, in Proc. ICIP'97, 26-29 Oct. 1997, vol. 2, pp. 795-798) suggested to obtain a displaced frame difference from two previously decoded images based on which the motion compensation methods were switched. In particular, a decoder with no extra information was made to adaptively select between the block motion compensation and the overlapped block motion compensation by allowing the overlapped block motion compensation to be performed only when a differential image block at the same location of the current block for being motion compensated has a number of pixels above a particular threshold. Likewise, other various conventional techniques have their proprietary criteria to obviate extra information needed to select the motion compensation methods for each block, and Young Su Lee suggested in “Moving picture compensation apparatus in MPEG4 decoding”, Korean Patent Registration No. 1002804980000, Nov. 10, 2000 to use a high band frequency energy value of transmitted and decoded DCT (discrete cosine transform) coefficients; Sung-hee Lee and Bong-soo Hur (“Apparatus to provide block-based motion compensation and method thereof”, US Patent, PN US2004/0252896, Dec. 16, 2004), Seung Hwan Kim, Dong-il Chang, Choong Woong Lee, and Sang Uk Lee (“Complexity reduction method for overlapped block motion compensation based on spatio-temporal correlation”, in Proc. ISCAS'99, 30 May-2 June 1999, vol. 4, pp. 211-214), and others suggested using motion vector differences between a current block under decoding and its neighboring blocks as the criteria; and Jun Zhang (“Adaptive overlapped block matching for accurate motion compensation”, US Patent, PN US2006/0083310, Apr. 20, 2006) selectively applies an overlapped block motion compensation based on the statistical standard deviation between inside and outside regions of a block for the motion compensation.
Though the above prior arts for adaptively selecting motion compensation methods without requiring extra information have contributed to relieving from the over-smoothing problem by providing their respective selection criteria, they commonly have one important deficiency. That is, the respective approaches select a motion reconstruction method in units of respective blocks methods and let the selected method applied to performing motion reconstructions on all of the pixels within a block, which disables a motion reconstruction in a finer unit such as on the basis of boundaries of the respective blocks or by each pixel in a block resulting in incomplete solutions to the over-smoothing problem.
This deficiency may be somewhat resolved by a method suggested by Jiro Katto (“Overlapped motion compensation using a window function which varies in response to an input picture”, U.S. Pat. No. 5,602,593, Feb. 11, 1997) wherein a weight for use in the weighted addition is adjusted by an input image and the statistical characteristics of a motion compensation error. In other words, the value of the weight for use in the overlapped block motion compensation is made variable by the statistical characteristics of a particular video inputted or a particular image of the inputted video, or even a particular boundary of the inputted image with the weight valued ‘0’ directing the typical block motion compensation method, which entitles the suggested method to be an expansion of the conventional block based adaptive motion compensation method for selecting motion reconstruction methods to an even more refined level at the block boundary units. However, since the statistical characteristics referred to in this method as being the basis for adjusting the weight mean the inputted images, or a statistical standard deviation and correlation belonged to a group of pixels corresponding to a part of an image, and a statistic expectation of the motion reconstruction errors, there is a significant computation volume required for their estimations, and its application to such a small region as a block boundary increases the uncertainty of the estimation accuracy resulting in a limitation of the performance. Besides, such statistical parameters used in the weight adjustment are not what can be reproduced by calculations in a decoder and they must be disadvantageously transmitted in the form of extra information to the decoder.
Problems with the suggested method by Jiro Katto (“Overlapped motion compensation using a window function which varies in response to an input picture”, U.S. Pat. No. 5,602,593, Feb. 11, 1997) are expected to be resolved by a modification of the suggestion of Byeong-Doo Choi, Jong-Woo Han, Chang-Su Kim, and Sung-Jae Ko (“Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation”, IEEE Trans. Circuits and Syst. for Video Technol., vol. 17, pp. 407-416, April 2007) which is based on a reliability for indicating the similarity of reconstructions of pixels of the subject block using motions of boundary blocks to the actual pixels of the block for current reconstruction and uses a greater weight in proportion to the reliability. However, because this method is designed for application of frame interpolation, the reliability information in the video compression application cannot be reproduced in a decoder as the weight adjustment criterion, and the method is unable to expand towards the overlapped block motion compensation by the pixel units short of becoming a more refined type of the adaptive technique.
An example of a conventional method for adjusting the overlapped block motion compensation by the pixel units was suggested by Chih-lung Bruce Lin, Ming-Chieh Lee, and Wei-ge Chen (“Overlapped motion compensation for object coding”, U.S. Pat. No. 5,982,438, Nov. 9, 1999) wherein the respective pixels in a single block are subjected to the overlapped block motion compensation only when they belong to the same object as that of the adjacent blocks whereby the over-smoothing problem is resolved. However, this method needs a provision of an object map and a transmission as additional information of the object map, which is hardly generated by the current techniques automatically from natural images which renders it really impractical for compressed encoding of video information.